Papers with math tasks
Solving for X and Beyond: Can Large Language Models Solve Complex Math Problems with More-Than-Two Unknowns? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks for Large Language Models often feature simple problems with only one or two unknown variables, which do not sufficiently challenge their reasoning capacities. |
| Approach: | They propose a new benchmark, BeyondX, which progressively increases complexity by expanding the number of unknowns in simpler problems. |
| Outcome: | The proposed approach improves performance on the BeyondX benchmark and provides deeper insights into the computational limits of LLMs when faced with more complex mathematical challenges. |
Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) achieve impressive performance on complex benchmarks yet sometimes fail on basic math reasoning. |
| Approach: | They propose a benchmark to evaluate the efficiency of reasoning in large language models . they formalize the accuracy-verbosity tradeoff and introduce the overthinking score . |
| Outcome: | The proposed model performs well on complex benchmarks but fails on basic math reasoning . the proposed model generates 18 more tokens while achieving lower accuracy . |
PyraMathBench: Evaluating and Improving Mathematical Capability in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Numerical reasoning is ubiquitous in scientific research and financial analysis, but few benchmarks evaluate them by integrating numerical processing and mathematical reasoning. |
| Approach: | They propose a numerically-integrated hierarchical benchmark with 27,215 questions derived from 7,404 math word problems that spans 4 key cognitive aspects, 14 subcategories, and 2 modalities. |
| Outcome: | The proposed model improves Qwen-2.5 score with SOLVE and IRPO training. |